Holography-based device, system and method for coded aperture imaging

ABSTRACT

A system and related method for coded aperture sensing, comprising: passing at least one scene wavefront from a target scene through at least one original coded aperture mask onto a focal plane array, producing a diffracted projection of the target scene; and processing the diffracted projection into a representation of the target scene by correlating a function of the diffracted projection with a function of a known array pattern of the at least one original coded aperture mask and by using at least one reconstructing wavefront for holographic reconstructing.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims benefit of U.S. Provisional application61/123,371 filed Apr. 7, 2008.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was supported, in whole or in part, by contractsW911QX09C0039 from the United States Army RDECOMAC, N68335-08-C-0557from the United States Naval Air Warfare Center, and FA9453-09-M-0103from the United States Air Force Research Laboratory. The Government hascertain rights in the invention.

FIELD OF THE INVENTION

The invention relates to coded-aperture imaging, in particular todiffractive coded-aperture imaging which can be effectively applied inspectral ranges where diffraction may have significant effect, includingultraviolet through infrared and longer-wavelength spectral ranges.

BACKGROUND OF THE INVENTION

Coded apertures have been used in astronomical imaging for a number ofyears. In these applications, imaging is performed in very short-wavespectral ranges, such as X-rays, where diffraction is negligible.

When coded-aperture imaging is attempted in the infrared spectral range,the effect of diffraction is significant and may pose a problem. Suchimaging in the presence of diffraction is of practical interest. (TimClark and Esko Jaska, DARPA Interest in Diffractive Sensors, Proc. ofSPIE Vol. 6714, 671403, (2007))

Diffraction causes a blur in image on the focal plane array (FPA).Slinger et al. used an approach to deblurring the image in the processof coded-aperture decoding, with Fourier deconvolution and noisereduction by Tikhonov regularization on multiple captured FPA frames (C.W. Slinger, M. Eismann, N. Gordon, K. Lewis, G. McDonald, M. McNie, D.Payne, K. Ridley, M. Strens, G. De Villiers, R. Wilson, An investigationof the potential for the use of a high resolution adaptive codedaperture system in the mid-wave infrared,” Proc SPIE 6714-07 (2007) andC. W. Slinger, G. D. De Villiers, D. A. Payne, Processing method forcoded aperture imaging, WO 2007/091049).

A different approach disclosed in a PCT publication by Slinger (C. W.Slinger, Imaging System, WO 2007/091051) uses a coded diffractive maskdesigned such that its diffraction pattern at the waveband of interestis a well-conditioned coded intensity pattern having a strongautocorrelation function with low sidelobes. Radiation reaching thedetector array is diffracted by the diffractive mask but in a definedway, and it is the diffraction pattern of the mask which provides thecoding. The scene image can then be reconstructed using the sametechniques as for conventional coded aperture imaging but using thediffraction pattern of the mask as the aperture function. Thediffractive coded aperture mask in Slinger's invention acts in effect asa hologram that is reconstructed by a plane wavefront from a distanttarget to produce a well-resolved coded pattern on the FPA. Thewell-resolved pattern is a traditional coded aperture pattern. The codedpattern on the FPA is then processed the same way as X-ray and other“diffraction-free” coded-aperture images.

It is known from holography that an aberration-free reconstructed imagecan only be produced when the reconstructing wavefront is of exactly thesame nature as the reference wavefront that was used during the hologramrecording. For example, if a plane wavefront is used as the referencefor recording, a plane wavefront of exactly the same orientation duringreconstruction is required to produce an aberration free image. If thereconstructing wavefront arrives at the hologram at a different angle,the image will be aberrated. This limits the field of view of theinvention disclosed in WO 2007/091051, where different points of thefield will produce different “reconstructing wavefronts” of thecoded-aperture mask “hologram.” Only for one look angle can the“hologram” mask be designed to produce an aberration-free array image onthe FPA. At other look angles, aberrations may be significant,increasing with the deviation of the look angle from the design value,as well as with the aperture size of the “hologram” array mask.

Slinger's PCT publications WO/2007/091049, WO2007/091047, WO2006/125975,and WO/2007/091051 disclose an imaging system where the coded-aperturemask acts as a diffractive optical element, or hologram, so thatradiation reaching the FPA, diffracted by the mask, produces awell-conditioned intensity pattern on the FPA, with strongautocorrelation and low sidelobes. As between 1) feature size of thediffracted pattern on the FPA and 2) FPA pixel pitch, the larger(coarser) of these two determines the angular resolution.

Slinger's invention is prone to aberrations at look angles substantiallydifferent from the look angle for which the diffractive mask isdesigned. For every different look angle, Slinger's imaging system inprinciple requires changing the diffractive pattern on the mask.Slinger's inventions produce intensity images; sensing the phase of anarriving wavefront is not provided.

Slinger's diffractive masks, due to their binary nature (transparentversus opaque, bit depth of 1), produce higher diffraction orders withnoise-like stationary image artifacts. The use of low-noise sensors doesnot reduce this detrimental effect. This problem is mitigated bycapturing and statistical treatment of multiple images of the samescene, with different, dynamically changing, adaptive mask patterns.This mitigation, however, requires complex adaptive masks (e.g., usingmicro-electromechanical, MEMS, devices), stationary objects that do notmove between frames, and a stable platform on which the imaging systemis installed.

Slinger's imaging, as described in these patent publications, also doesnot provide color information.

It is an object of this invention to provide a coded aperture imagingsystem with high imaging resolution in the visible and infrared spectralranges, where there may be significant diffraction.

Another object of this invention is to achieve incident wavefrontsensing, including amplitude and phase.

Another object of this invention is to remove aberrations caused bywindows or optics present in the optical path preceding the aperture ofthe system.

Another object of this invention is to remove aberrations caused byatmospheric turbulence in the optical path preceding the aperture of thesystem.

Another object of this invention is to provide synthetic-apertureimaging with multiple sensing apertures jointly forming a highresolution image of a scene.

Another object of this invention is to provide a coded-aperture imagingsystem operable in the ultraviolet, visible, infrared, andlonger-wavelength spectral ranges with a simplified coded aperture mask.

Another object of this invention is aberration-free imaging, overextremely wide fields of view, with a single coded-aperture mask.

Another object of this invention is to achieve diffractivecoded-aperture imaging free of noise-like image artifacts.

Another object of this invention is to provide color, spectral, andhyperspectral sensing and imaging.

Another object of this invention is to provide three-dimensional imagingof objects.

Another object of this invention is to provide polarization-sensitiveimaging and wavefront sensing.

Finally, another object of this invention is to provide coded-apertureimaging with automated detection of change.

SUMMARY OF THE INVENTION

This invention is a method of coded-aperture imaging which can be usedeffectively in the ultraviolet through infrared and longer-wavelengthspectral ranges, in the presence of diffraction. The pattern formed byradiation diffracted on a coded-aperture mask, of a structure similar tothat used in “diffraction-free” coded aperture imaging, is processed asa digital hologram, by means of digital holography. Thedigital-holographic processing results in a reconstructed wavefrontwhich yields a high-resolution image of the scene.

In particular, what is disclosed is a system and related method forcoded aperture sensing, comprising: passing at least one scene wavefrontfrom a target scene through at least one original coded aperture maskonto a focal plane array, producing a diffracted projection of thetarget scene; and processing the diffracted projection into arepresentation of the target scene by correlating a function of thediffracted projection with a function of a known array pattern of the atleast one original coded aperture mask and by using at least onereconstructing wavefront for holographic reconstructing.

BRIEF DESCRIPTION OF THE DRAWINGS

The features of the invention believed to be novel are set forth in theappended claims. The invention, however, together with further objectsand advantages thereof, may best be understood by reference to thefollowing description taken in conjunction with the accompanyingdrawing(s) summarized below.

FIG. 1 is a schematic illustration of a basic layout of coded-apertureimaging in accordance with various embodiments of the invention.

FIG. 2 is a schematic illustration of holographic recording of anincident plane wavefront.

FIG. 3 is a schematic illustration of holographic reconstruction of anincident plane wavefront.

FIG. 4 is a schematic illustration of holographic a) recording of anincident tilted wavefront; and b) reconstruction of the tiltedwavefront.

FIG. 5 is a schematic illustration of an alternative holographicreconstruction.

FIG. 6 is a schematic illustration of holography with a finite sizeobscuration and complementary aperture: a) recording; b) reconstruction.

FIG. 7 is a flowchart of a preferred embodiment of the coded-apertureimaging process according to the invention.

FIG. 8 is a flowchart of a first alternative embodiment for thecoded-aperture imaging process.

FIG. 9 is a flowchart of a second alternative embodiment for thecoded-aperture imaging process.

FIG. 10 is a flowchart of a third alternative embodiment for thecoded-aperture imaging process.

FIG. 11 is a flowchart of a fourth alternative embodiment for thecoded-aperture imaging process.

FIG. 12 is a plan view of a) the physical layout of coded aperture maskand FPA; b) effective holographic reconstruction after correlation.

FIG. 13 is a schematic view of “Twin” reconstructed wavefronts.

FIG. 14 is an image illustrating how the frequency of the fringes nearthe edge of FPA reaches the pixel spatial frequency. A second FPA istilted so that fringes beyond the first FPA are still resolved.

FIG. 15 is a plan view illustrating, for large coded-aperture masks, alens may be used to direct more light to the FPA and the “holographiclens,” enabling the use of larger masks. It is not the purpose of thislens to produce a focused image.

FIG. 16 is a plan view illustrating how a curved coded-aperture mask incombination with multiple FPAs or a curved FPAs offers extended field ofview coverage, including conformal imaging.

FIG. 17 is a schematic illustration of Fourier image processing forchange detection.

FIG. 18 is an image illustrating numeric simulation of change detection.

FIG. 19 contains a set of images illustrating numerical modeling ofcoded-aperture imaging according to various embodiments, specifically:a) graphical user interface (GUI) showing a model object, coded-aperturemask, and diffraction pattern on the FPA; b) the same, diffractionpattern correlated with the complementary mask pattern; c) areconstructed image of the object; d) a fragment of correlation (b); ande) a fragment of reconstruction (c).

FIG. 20 contains a set of images illustrating diffractive coded-apertureimaging of transparency “A”, specifically: a) a diffraction on the FPAfrom a “positive” coded-aperture mask; b) the same with complementarymask; c) the difference of the foregoing two diffraction patterns; d)the coded-aperture mask; e) the digital hologram—correlation of thedifferential diffraction pattern with the mask; and f) the output ofdigital reconstruction of the hologram.

FIG. 21 contains a set of images illustrating diffractive coded-apertureimaging of transparency “F”, specifically: a) a diffraction on FPA froma “positive” coded-aperture mask; b) the same with complementary mask;c) the difference of the foregoing two diffraction patterns; d) thecoded-aperture mask; e) the digital hologram—correlation of thedifferential diffraction pattern with the mask; and f) the output ofdigital reconstruction of the hologram.

FIG. 22 contains images illustrating single-frame reconstructionscomparable to differential-frame reconstructions of FIGS. 20 and 21.

FIG. 23 illustrates the polarization-based separation of diffractions byoriginal and complementary patterns of the mask using a polarizingbeamsplitter.

FIG. 24 illustrates the polarization-based separation of diffractions byoriginal and complementary patterns of the mask using a (voltagecontrolled) polarization rotator.

FIG. 25 depicts a polarization coded-aperture mask comprising fourdifferent types of surface areas.

DETAILED DESCRIPTION

Operating Principle

FIG. 1 illustrates the basic layout of diffractive coded-apertureimaging. Light 11 from the scene arrives at the coded-aperture mask 12.If the scene is distant, its every point corresponds to an incidentplane wavefront. If the scene is at a finite distance, the wavefrontsare spherical. The incident light is diffracted on the coded-aperturemask 12. The diffracted light 13 is registered by the focal plane array(FPA) 14. The coded-aperture mask 12 may be adaptive, so that theaperture array may be controlled by a suitable controller 15. In sum: atleast one scene wavefront from a target scene is passed through at leastone coded aperture mask onto a focal plane array, producing a diffractedprojection of the target scene.

The technical challenge addressed by this disclosure, is how to thenmake use of this diffracted projection to reconstruct the image of thetarget scene, especially for long wavelengths where diffraction may havea significant impact.

Introduction

This invention provides for imaging and wavefront sensing using acoded-aperture mask 12 which may include, without limitation, auniformly redundant array similar to those used in coded-apertureimaging in astronomy (see, e.g., A. Busboom, H. Elders-Boll and H. D.Schotten, Uniformly Redundant Arrays, Experimental Astronomy, Volume 8,Number 2 (June 1998) pp. 97-123). Diffraction of light on thecoded-aperture mask 12 array produces a pattern on a focal plane array14. One or more sensed FPA patterns are captured and processed usingholographic methods. The result of the processing is a high-resolution,diffraction-limited image of the scene and a reconstruction of thewavefront from the scene, including phase information.

This is in contrast to, for example, the Slinger methods, which replacethe “traditional” coded aperture mask with a diffractive optical element(hologram) that would, for each point of a distant scene (planewavefront, or “star”), produce the image of the traditional codedaperture mask on the sensor. In the invention disclosed herein, the“traditional” coded aperture mask is substantially retained, andinstead, the image falling on the focal plane array 14 is treated as adigital hologram.

Following is a simplified, introductory illustration of the operatingprinciple of this invention. Referring to FIG. 2, consider an aperturemask 21 in the shape of a small obscuration (“dust particle”). Theincident light 11 is taken to be an incident plane wavefront from apoint of the distant scene to be recorded (target scene, not explicitlyshown), and this is scattered by the small obscuration, forming ascattered spherical wavefront 22. Most of the plane wavefront is notscattered, and instead propagates to the FPA 14 where it interferes withthe scattered spherical wave. The result of the interference is thewell-known concentric, intermittently dark and bright, Fresnel rings.The rings are spaced more sparsely near the center and are increasinglymore frequent as the distance from the center increases. If the fringes23 are registered on a photographic plate, the processed photographicplate, thereafter illuminated by an incident light, will act as adiffractive optical element, or holographic lens 24. If the fringes 23are digitally recorded, then they can be used to represent a “virtual”holographic lens. Both approaches are envisioned here in variousembodiments: use of real, physical optical elements and processes, anduse of “virtual,” digitally-represented optical elements and processes.

Referring to FIG. 3 with the incident light now taken to be an incidentplane reconstructing wavefront 35 (which need not emanate from the scenebeing recorded and which may be virtually, digitally simulated), nowconsider an opaque screen or mask 31 with a small pinhole 32. The screenor mask 31 is complementary to the small obscuration 21 of FIG. 2, andis placed at the same distance from the FPA. That is, the space throughwhich the incident light freely propagates in FIG. 2 is occupied byscreen or mask 31 in FIG. 3, and the pinhole 32 through which the lightpropagates in FIG. 3 is occupied by the small 21 obscuration of FIG. 2.In other words, screen or mask 31 and small obscuration 21 areeffectively the “negatives” of one another. The (real or virtual)reconstructing wavefront 35 passing through the pinhole forms aspherical wavefront 33, which is then collimated by the (real orvirtual) holographic lens 24 that resulted from FIG. 2, into areconstructed plane wavefront 34 which will turn out to be identical tothe original incident plane wave of FIG. 2. Alternatively, the patterncaptured by the FPA 14 can be digitally, virtually represented andilluminated by a spherical wave to digitally reconstruct the wavefront.The digitally-reconstructed wavefront will contain the proper phaseinformation (tilt) of the original plane wavefront 11 from the targetscene.

More generally, screen or mask 31 is a “function of” the known smallobscuration 21, and in the illustration, this “function” is the“complement,” which is the most straightforward function to employ andso is preferred but is not limiting. All that really is required,however, is that this function be a function other than the identifyfunction, i.e., that 21 and 31 be known, different functions of oneanother.

If the angle of incidence of the plane wavefront at recording changes(e.g., because it is arriving from a different point of the scene),after reconstruction, the change in the wavefront tilt is faithfullyreproduced.

With this basic illustration in mind, a complex scene can be imaged inthe same manner, since each point of the scene will be represented by aplane wavefront arriving at a unique angle, see FIG. 4.

If the target is at a finite distance, the above discussion remainsvalid, except that the captured and restored wavefront will bespherical, centered on a point of the target. As such, it does notmatter if the object being imaged, i.e., target scene, is near or far.

The photographic plate can be replaced by a two-dimensional imagingsensor, or a focal plane array (FPA), including CCD, CMOS, or any othertype of imaging sensor known or which may become known in the art tothose of ordinary skill. The FPA may capture the holographic fringepattern, producing its digital representation. The captured pattern(hologram) may be displayed on a spatial light modulator (SLM) andrestored optically, using a laser. Alternatively and preferably, thehologram captured by the FPA may be reconstructed digitally, usingestablished means of digital holography known in the art. Any of theexisting algorithms known or which may become known in digitalholography may be used for this reconstruction, including, withoutlimitation, digital Fresnel propagation.

The above simplified example illustrates much of the basic operation ofthis invention: recording a diffraction pattern (hologram) produced bythe light arriving from the scene and passing through a coded-aperturemask, which arrives at the FPA as a diffracted projection of the targetscene, followed by reconstruction of the resulting hologram with adifferent (physical or digital) reconstructing wavefront, resulting froma mask that is complementary to (and more generally a known function of)the original coded-aperture mask used in recording the hologram.

Shown in FIGS. 2 and 3 are the incident wavefronts parallel to the FPA.If the wavefront is incident at an angle as in FIG. 4, the concentricfringes will shift on the FPA and change shape slightly. Still, atreconstruction, the spherical wave from the same pinhole of thecomplementary mask will be collimated by the hologram into anaberration-free plane wavefront, identical to the original incidentwavefront which produced the hologram.

A complex wavefront arriving from the target scene can be presented as asum of component plane wavefronts, each arriving from a point of thetarget scene. The above recording with a “dust particle” mask andreconstruction with the complementary pinhole mask will faithfullyrestore the entire complex wavefront.

Mathematically, the process of recording and reconstructing thecoded-aperture hologram, and why this invention works, can be describedas follows. Let the wavefront from the scene or target in the absence ofthe coded-aperture mask be U₀. U₁ and U₂ are the wavefronts afterpassing the “original” and “complementary” coded-aperture masks,respectively. Accordingly, the following holds true, for all spatiallocations:U ₀ =U ₁ +U ₂  (1)

The FPA registers intensity of the incident light, in the FPA plane,given by:I ₁=(|U ₁|)²  (2)

On the other hand, the same intensity can be expressed in terms of thefield after the complementary mask:I ₁=(|U ₀ −U ₂|)²=(|U ₀|)² +U ₀· U ₂ + U ₀ ·U ₂+(|U ₂|)²  (3)Here,Ū=Re(U)−i·Im(U)  (4)is a complex conjugate.

The second and third terms of (3) are the “holographic” terms, capturingthe incident wavefront U₀.

In the above discussion, relationships (1) and (3) are not trivial.Knowing the field at the FPA, diffracted by the complementary mask,which in turn, at reconstruction, defines diffraction on the hologram,requires knowing the wavefront from the target, U₀. In other words,knowing three patterns: 1) the diffraction pattern on the FPA with theoriginal mask; 2) the diffraction pattern on the FPA from thecomplementary mask; and 3) the mask pattern, is equivalent to knowingthree other values: 1) the phase, or tilt, of the wavefront from thetarget; 2) the diffraction pattern at the FPA from original mask; and 3)the mask pattern.

According to the above discussion, the intensity pattern on the FPA 14is treated as a digital hologram, which is the result of interference oftwo wavefronts: 1) the wavefront arriving from the target scene,unobstructed, as if no mask was present between the target scene and theFPA 14; and 2) the same wavefront, modulated by the complementary mask,which is identical to the unobstructed wavefront from the scene at thetransparent points of the original mask and completely absorbed at theopaque points of the complementary mask. The complementary mask ispreferably the negative of the physical mask used in forming thephysical intensity pattern on the FPA, i.e., the complementary mask isopaque in the points where the physical mask is transparent, and viceversa.

Reconstruction of the digital hologram by an unobstructed wavefrontproduces the wavefront (complex, with intensity and phase reconstructed)modulated by the complementary mask. Reconstruction of the digitalhologram by the wavefront modulated by the complementary mask producesthe unobstructed wavefront (complex, with intensity and phasereconstructed).

The patterns of the original and complementary masks are not limited toa single pinhole and single “dust particle,” which are merelyillustrative, simplified examples. The above discussion is equallyapplicable to any pattern of the mask.

Preferred Reconstruction

Preferred reconstruction is performed as illustrated by FIG. 4, whichshows the incident light 11 from the target scene entering at a tilt inFIG. 4 a. The reconstructing wavefront 35, after diffraction on theaperture of mask 31, becomes spherical wavefront 33, with the centerlocated on-axis, at the distance equal to the distance between thecoded-aperture masks 21, 31 and the FPA 14. The axis is defined as aline from the center of the FPA and normal to the FPA, pointing at thecenter of the scene, and so is horizontal in FIG. 4. For the point inthe center of the scene, the reconstruction is shown in FIG. 4 b. For adifferent point, the “holographic lens” created during recording will bede-centered, i.e., shifted, as illustrated, and its shape will differfrom circular. The reconstructed wavefront will be plane,aberration-free, and tilted exactly as the original wavefront from thepoint of the scene.

Forming an image from the reconstructed plane wavefronts is performedwith a “digital perfect lens” to digitally focus each plane wavefrontinto a point. Since the plane wavefronts are reconstructed with noaberrations, the resulting image is diffraction-free as well.

The reconstruction can be mathematically described as multiplication ofthe FPA intensity pattern by the complex field of the reconstructingwave, which is the field behind the (pinhole) mask complementary to theobscuration (“dust particle”) mask used during recording (see equation(3)):I ₁ ·U ₂=(|U ₀|)² ·U ₂ +U ₀· U ₂ ·U ₂+ U ₀ ·U ₂ ·U ₂+(|U ₂|)² ·U ₂  (5)

The second term on the right-hand side is the reconstructed originalwavefront (see equation (4)):U ₀· U ₂ ·U ₂ =U ₀·(|U ₂|)²  (6)

It is the complex field U₀ at the FPA, multiplied by the scalar(intensity) value (|U₂|)² of the illumination of the FPA by thereconstructing wave, which is the field behind the complementary mask,U₂. The first and the last terms in (5) are for zero-order diffractionon the holographic lens; the third term is the “twin” image.

The last, fourth, term (|U₂|)²·U₂, zero-order diffraction, creates ablurred projection of the target, convolved with the elementary“building block” of the mask and convolved with the propagation function(diffraction from the mask to the FPA). This blurred image of the scenewill be located on the hologram. The hologram during reconstruction willact as a collimating lens; the blurred image of the scene from thisfourth term of (5) will act as a background illumination.

Alternative Reconstruction

An alternative reconstruction is with a plane wavefront, and is shown inFIG. 5. The different “holographic lenses” produced by the wavefront ofdifferent tilts, arriving from different points of the scene, areshifted from the center of the FPA by different amounts. One planereconstructing wavefront 35, parallel to the FPA 14 (normal to the axis)will be focused by the different holographic lenses into differentpoints in the focal, or image, plane 51, as illustrated by the differentpoints 54 and 55. The drawback, compared to the preferred reconstructionwith the spherical wave discussed in connection with FIG. 4, is thatoff-axis lenses, produced by interference with a tilted wavefront duringrecording, are not only shifted 52 relative to the unshifted, on-axislenses 53, but also differ in shape from circular. As a result, the realimages 54 of the off-axis points will be aberrated—progressively so astheir deviation off-axis increases. The real image 55 of an on-axispoint is also illustrated.

A remedy is to use plane wavefronts of different tilts forreconstruction. In a range of angles close to the recording angle,aberrations will be small. Several reconstructions may cover the wholescene, to produce the image of the scene with low overall aberrations.Still, this approach is less attractive compared to the preferredone-step reconstruction with the spherical wavefront illustrated by FIG.4.

Finite Size Aperture

The preceding discussion was of the wavefront capture and reconstructionwith the mask at recording being a small obscuration (“dust particle”),and at reconstruction the complementary mask being a small pinhole. Inthis approximation, the hologram is the interference pattern of twowavefronts: spherical, scattered on the dust particle, and theundisturbed plane wavefront.

If the obscuration has a finite size, as now illustrated by FIG. 6, theabove considerations would need to be detailed as follows. At recording,the hologram is the result of interference between a wavefrontdiffracted on the obscuration and the plane wavefront disturbed by theobscuration that shadows it in part. At reconstruction, thereconstructing wavefront is produced by illuminating an opening in alarge opaque screen, the size and shape of the opening identical tothose of the obscuration used in recording. Therefore, the mask formingthe reconstructing wavefront is still complementary to the mask used forrecording. Accordingly, expressions (1)-(3) still apply.

The result of the reconstruction is the same wavefront as the planewavefront at recording: a nearly plane wavefront, disturbed by thepresence of obscuration in the optical path before the hologram, wherethe obscuration introduced by 21 was during the recording.

The “disturbance” of the reconstructed wavefront will not affect theimaging significantly. For each point of a distant scene or target, alarge wavefront will be reconstructed, over aperture sizes likely in thecentimeters or tens of centimeters range. The “disturbance” will be froman obscuration (a “virtual image” of the obscuration) the size of thecoded-aperture pixel (“building block”), likely, tens to hundreds ofmicrons. To produce the image of the scene, the collimated wavefront is(digitally) focused by a perfect lens into its focal plane. The effectof the wavefront disturbance is similar to that of a small obscurationat or before the pupil of a large lens—a negligible effect.

Explanation of Simplifications in Foregoing Discussion

FIGS. 2 and 3, as well as FIGS. 4 a and 4 b, as well as FIGS. 6 a and 6b, are simplified in two ways for purposes of illustration, in relationto the actual preferred practice of the invention. First, FIGS. 2, 4 aand 6 a illustrate a coded-aperture mask 21 which is a “dust particle”or “obscuration,” while FIGS. 3, 4 b, and 6 b illustrate a“complementary” mask 31 which is the “negative” of the foregoing, i.e.,an opaque surface with a “pinhole.” In actual practice, as will beelaborated further below, coded-aperture mask 21 comprises an array ofobscurations and apertures with a single autocorrelation peak, such asbut not limited to a random array or a uniformly redundant array (URA)or a modified uniformly redundant array (MURA). The complementary arrayis still the negative of this. In practice, both the array and thecomplementary array preferably each contain obscurations overapproximately 50% of their surface, and apertures over the remainingapproximately 50% of their surface, with each array/mask being thenegative of the other, and wherein either array/mask can be used as the“original array/mask,” with the “complementary array/mask” then beingdefined as the “negative” of the original array/mask. Thus, both theoriginal masks labeled 21 and the complementary masks labeled 31 in thedrawings are both instances of the coded-aperture mask 12 as illustratedin FIG. 1, and also, FIG. 12 a. The question of what version of 12 isdefined and used to be the array/mask 21 and what is therefore selectedto be the complement/negative array/mask 31, becomes an arbitrarychoice, but in effect, element 12 of FIG. 1 this some sense become“replicated” twice, once for the “original mask” 21 and once for the“complementary mask” 31 (and more generally, for the “second” mask whichis a “function of” the original mask and preferable its complement).

Second, while element 12 in FIG. 1 is in this sense “replicated” twice,once for the “mask” 21 and once for the “complementary mask” 31, thereplication does not extend to the issue of the incoming wavefront,which is represented as 11 in FIGS. 1, 2, 4 a and 6 a and 35 in FIGS. 3,4 b and 6. This gets to the distinction between recording of a scenewhich is illustrated by FIGS. 2, 4 a and 6 a, and the “reconstruction”(or, loosely, playback) of the scene which is illustrated by FIGS. 3, 4b and 6 b. In FIGS. 2, 4 a and 6 a, the incoming wavefront 11 representsreal, physical luminous radiation which does originate from the targetscene being recorded, and is passed through the (original,non-complementary) mask 21. For reconstruction, however, the wavefront,now labeled as 35, does not necessarily originate from the scene but isan independent light source used for reconstruction, and in fact, thisreconstruction wavefront 35 may, but does not have to be, real, physicalluminous radiation and, in the preferred embodiments, is in factvirtual, digitally-represented radiation. That is, in the preferredembodiment, FIGS. 2, 4 a and 6 are present real, physical processes, andFIGS. 3, 4 b and 6 b represent, in at least some embodiments, digital,virtual processes.

Finally, recognizing that the recording and reconstruction of theoriginal recorded target scene essentially involves using an originalmask and its complement, and separating various features of the imageusing equations (1) through (6) above to arrive at a faithfulreconstruction of the original scene, the reader may note, if the real,physical wavefront of the scene itself is passed through the mask as inFIGS. 2, 4 a and 6 onto the (original) focal plane 14 but a different(real or virtual) wavefront is passed through the complementary mask asin FIGS. 3, 4 b and 6 b onto another (complementary) focal plane, thatsome information from the scene could be lost. This is because equation(1) assumes that U₁ is the wavefront from the scene, which it is, andthat that U₂ is also the wavefront from the scene, which, in at leastsome embodiments, it is not. Rather, U₂, in some of the embodiments, isan independent, real or virtual, reconstructive wavefront. It should benoted that in these embodiments, there is in fact some information losswhich occurs because U₂ is not a wavefront from the scene. However, theonly information lost in these embodiments is certain phase informationwhich does not affect the ability to achieve a precise reconstruction ofthe original image. In the event an application of this invention doesrequire this missing phase information, there are further techniqueswhich can be applied to restore this information as well, although thesetechniques are beyond the scope of the present disclosure.

Coded Aperture Array

The above consideration was of a mask produced by a single obscurationand the complementary mask—single aperture. The same consideration,including expressions (1)-(3), is valid for any array of apertures,complementary to an array of obscurations. The two-dimensional arrayneeds to have a single autocorrelation peak. This condition issatisfied, for example, by a random array. Another satisfactory exampleis a uniformly redundant array (URA) or a modified uniformly redundantarray (MURA).

Each of the “dust particles” or obscurations of the array, illuminatedby a wavefront from a point of the target, will produce a “holographiclens” as was already discussed. Each “holographic lens” is a result ofrecording a hologram of the light diffracted on the “dust particle”interfering with the wavefront un-diffracted on the particle, i.e.,near-circular fringes forming the holographic lens. Reconstruction ofthe hologram is done with the “complementary mask” with a pinhole inplace of each “dust particle” (opaque point of the original mask). Thespherical wavefront from each pinhole is collimated by the respectiveholographic lens to reproduce the same wavefront as the undisturbedwavefront at recording, for any tilt of the wavefront.

Many “dust particles” placed at random to forming URA, act similarly toone particle/pinhole. For any wavefront, un-diffracted light willinterfere with the light scattered on the given particle as if therewere no other particles. As for the interference between the lightscattered by the given particle and the other particles, the phase willbe random, canceling out the contribution of this interference duringcorrelation with the mask, as is now discussed in the following:

The diffraction pattern captured by the FPA is correlated with thecomplementary mask (pinholes) of the coded array. As a result, the many“holographic lenses” captured by the FPA form one “holographic lens” inthe output of the correlation processing in which cross-correlation ofthe FPA pattern and of the mask coded pattern is computed. The effect ofinterference of the many scattered spherical waves from each “dustparticle” with each other will be negligible due to the statisticalproperties of the mask having a single autocorrelation peak. This makesit possible to reconstruct all wavefronts from all points of a scene ortargets using one spherical reconstructing wavefront—provided the maskelement is sufficiently small—so that diffraction on the mask resultsessentially in a spherical wavefront. The reconstruction with a singlespherical wavefront is a convenience but not a requirement of thisinvention.

The mask element may be larger, and the diffracted wavefront may differfrom spherical, as long as the diffraction angle is large for thediffracted light to cover a substantial part of the FPA. In this case,reconstruction should be done with the wavefront diffracted on a singlemask element, not necessarily spherical. The result of thereconstruction will still be the sensed wavefront arriving from thescene at the mask and the diffraction-limited image of the scene.

Even if the mask element is too large to cause significant diffraction,the processing according to this invention will produce an image of thescene, with the resolution being limited by the FPA pixel size, as inthe prior art, rather than being diffraction limited.

Unlike the inventions disclosed in the patents to Slinger, thisinvention uses what Slinger refers to as standard coded aperture mask.Unlike Slinger, who uses the aperture mask as the hologram to form awell-defined pattern (similar to a standard coded aperture mask) on theFPA, this invention forms a digital hologram on the FPA, produced bydiffraction on a standard coded aperture mask. In other words, Slinger's“holographic mask” forms a “standard coded aperture” pattern on thefocal plane array, whereas this invention uses a “standard codedaperture mask” to form a “digital hologram” on the focal plane array.

This invention then uses the digital hologram formed on the FPA, withmany levels of gray and bit depth defined by the FPA, as the basisholographically reconstructing the original target scene image. As aresult, the higher order diffraction at reconstruction is not an issue.Noise can be reduced with low-noise and high bit-depth sensors (FPA).Single-frame sensing allows for conventional, passive, low-cost masksand for imaging from moving and vibrating platforms.

All embodiments entail passing at least one scene wavefront from atarget scene (11) through at least one original coded aperture mask (12,21) onto a focal plane array 14, producing a diffracted projection 24 ofthe target scene; and processing the diffracted projection 24 into arepresentation of the target scene by correlating a function of thediffracted projection (often, but not always, the diffracted projectionsitself) with a function of a known array pattern of the at least oneoriginal coded aperture mask (preferably, but not limited to, thecomplement of the original mask) and by using at least onereconstructing wavefront 35 for holographic reconstructing.

In the following, a two-dimensional digital pattern of values −1corresponding to opaque, and +1 corresponding to transparent areas of amask, is referred to as a bipolar pattern. Referring to FIG. 7, thepractical preferred embodiment method is as follows:

-   -   1. Place an FPA behind a coded-aperture mask with a coded        pattern of apertures and obscurations—the mask preferably, but        not limited to, a URA. The feature size on the mask is        preferably small. The mask is illuminated by light arriving from        the target and/or scene.    -   2. Capture the raw diffraction pattern on the FPA, 71, resulting        in a mask-convolved, diffraction-blurred projection of the        target scene 72.    -   3. Calculate cross-correlation of the FPA pattern with the        complementary bipolar coded aperture mask pattern, 73. The        result is a hologram with a single “holographic lens” for each        point of the scene or target, 74.    -   4. Perform wavefront reconstruction: digitally (i.e. virtually)        “illuminate”, e.g., by digital Fresnel propagation, the hologram        by a reconstructing wavefront. The reconstructing wavefront is        the result of illuminating by a plane wavefront, propagating        along the system optical axis, of an aperture of a certain shape        and size, located on-axis in the same plane where the coded        aperture mask was during the recording 75. The shape and size of        the apertures are identical and complementary to those of the        “building block” of the coded-aperture mask. The “building        block” is the feature, e.g., a square of a finite size, or a        small circle, etc., which is repeated in the coded aperture mask        to form its pattern. If the “building block” has sufficiently        small size, the reconstructing wavefront is a spherical        wavefront, centered on-axis in the plane of the coded mask. The        result of the reconstruction will be a (digitally reconstructed)        phase wavefront for every point of the scene or target, 76.    -   5. From the reconstructed wavefront, the image of the scene is        produced, by optional “digital focusing” and propagation to the        image plane 77, where intensity at each point forms the image        78. That is, this optional step entails focusing the holographic        reconstruction onto an image plane, producing an image of the        target scene.

In this embodiment, the correlating occurs before the holographicreconstructing; the correlating produces a hologram of the target scene;and the holographic reconstructing comprises Fresnel illuminating thehologram into a holographic reconstruction of the target scene.

Two options for improving the reconstruction by implementing thecorrelation (step 3) in situations where the wavefront from a point ofthe target scene incident on the coded-aperture mask has aberrations oris otherwise substantially different from a plane wave over an areacomparable to the area of the FPA are as follows:

3, A) (alternative A to step 3) Calculate cross-correlation of afunction of the diffracted projection on the FPA, defined as a portionof the FPA pattern, with the complementary bipolar coded aperture maskpattern, 73. The result is a hologram with a single “holographic lens”for each point of the scene or target, 74.

-   -   3, B) (alternative B to step 3) Calculate cross-correlation of        the FPA pattern with a function of the complementary bipolar        coded aperture mask pattern, 73, namely, a morphed pattern of        the complementary bipolar coded aperture mask pattern. The        result is a hologram with a single “holographic lens” for each        point of the scene or target, 74. For example, if a lens is        present in the optical path, the lens transforming a plane wave        from a point of the scene into a spherical wave, the        above-mentioned morphing of the mask pattern is magnification,        or change of scale.

The foregoing functions of the diffracted projection and of the originalcoded aperture mask are applicable in other embodiments as well.

An option for implementing reconstruction (step 4) is to performphysical reconstruction instead of one which is digital, i.e., virtual.For the physical reconstruction, the hologram pattern is displayed on aspatial light modulator and illuminated by (preferably) a laser beam.

An optional processing step, to follow preferably after the FPA capture,is to subtract another FPA pattern, captured with the same scene and acomplementary coded-aperture mask:

2a) Capture the diffraction pattern on the FPA with the coded-aperturemask replaced by the complementary mask pattern, at 711, yielding asecond, complementary diffracted projection of the target scene 712.

2b) Subtract the two captured FPA patterns 713 to arrive at a diffractedprojection comprising a difference between the diffracted projection,and the complementary diffracted projection, 714. Use the difference inthe subsequent steps.

That is, at least one scene wavefront is passed from a target scenethrough a complement of the at least one original coded aperture mask,producing a complementary diffracted projection of the target scene;wherein: the function of the diffracted projection comprises adifference between the diffracted projection, and the complementarydiffracted projection.

This optional processing eliminates the last term in equation (5),(|U₂|)²·U₂, zero-order diffraction. Unlike the first term in equation(5), which is uniform illumination of the FPA by the distant scene, thislast term is the pattern of diffraction on the complementary mask.Depending on the mask feature size, this term may change relatively fastover the hologram, affecting reconstruction. The optional subtractionstep eliminates this effect. The associated cost is the need toco-register two captured frames and a doubled required bandwidth of thevideo channel. The processing overhead increases insignificantly,limited to frame subtraction.

The above discussion was for a single point of a distant target sceneand the respective single plane arriving wavefront. The same is true fora target scene point (object, target) at a closer distance, with theonly change being from plane wavefront to spherical wavefront centeredon the point of the target scene (which may be an object).

The above discussion can also be directly applied to any number ofpoints in the target scene, including continuous intensity distributionsand arbitrary wavefront shapes. As is discussed in the section SpatialCoherence, under natural illumination, resolved points of the scene aretypically spatially incoherent. For this reason, the “holographiclenses” produced in the FPA by the light from each resolved point(element) of the scene will be intensity-additive, with no coherenteffect (no interference). All “holographic lenses” are recorded with themaximum intensity in the center, so there is no “phase shift” betweenintensity patterns and the resulting holographic lenses. The reason forthis is that at recording, the central point of the “lens” has no phaseshift relative to the reference, un-diffracted, plane wavefront.

A first alternative embodiment, summarized in FIG. 8, is as follows:

-   -   1. Place an FPA behind a coded-aperture mask with a coded        pattern of apertures and obscurations—preferably, but not        limited to, a URA. The feature size on the mask is preferably        small. The mask is illuminated by light arriving from the target        scene.    -   2. Capture the raw diffraction pattern on the FPA, 71, resulting        in a mask-convolved, diffraction-blurred projection of the        target scene 72.    -   3. Calculate cross-correlation of the FPA pattern with the        complementary bipolar coded aperture mask pattern, 73. The        result is a hologram with a single “holographic lens” for each        point of the scene or target, 74. (the foregoing are identical        to the first three steps in FIG. 7.)    -   4. Perform wavefront reconstruction: digitally (i.e. virtually)        “illuminate”, e.g., by a plane wavefront parallel to the mask,        81. The result will be a set of (digitally reconstructed)        spherical wavefronts, 82, for each point of the scene.    -   5. Propagate (digitally, i.e., virtually) to focal plane 83;        calculate intensity at each point of the focal plane. The result        is the reconstructed image of the scene, 84.

To summarize, in this embodiment, correlating occurs before holographicreconstructing; the correlating produces a hologram of the target scene;and the holographic reconstructing comprises illuminating the holograminto a holographic reconstruction of the target scene; furthercomprising: propagating the holographic reconstruction onto an imageplane and calculating intensity at points of the image plane, producingan image of the target scene.

A second alternative embodiment, summarized in FIG. 9, is as follows:

-   -   1. Place an FPA behind a coded-aperture mask with a coded        pattern of apertures and obscurations—preferably, but not        limited to, URA. The feature size on the mask is preferably        small. The mask is illuminated by light arriving from the target        and/or scene.    -   2. Capture the raw diffraction pattern on the FPA, 71, resulting        in a mask-convolved, diffraction-blurred projection of the        target scene 72. (All as in FIGS. 7 and 8.)    -   3. (Digitally) illuminate the raw FPA pattern by a plane wave        and propagate by the mask-to-FPA distance 91. The result will be        a set of overlapping images of the mask pattern, each shifted by        a unique shift amount and direction for each point of the scene,        92.    -   4. Calculate cross-correlation of the result with the        complementary, preferably bipolar, coded-aperture pattern, 93.        The result will be the reconstructed image of the scene 94.

In this embodiment, holographic reconstructing occurs beforecorrelating; the holographic reconstructing comprises illuminating thediffracted projection of the target scene as a hologram of thediffracted projection to produce a pattern image; and the correlatingcomprises correlating the pattern image of the hologram with thefunction of the known array pattern of the at least one original codedaperture mask, producing an image of the target scene.

A third alternative embodiment, summarized in FIG. 10, is as follows:

-   -   1. Place an FPA behind a coded-aperture mask with a coded        pattern of apertures and obscurations—preferably, but not        limited to, URA. The feature size on the mask is preferably        small. The mask is illuminated by light arriving from the target        and/or scene.    -   2. Capture the raw diffraction pattern on the FPA, 71, resulting        in a mask-convolved, diffraction-blurred projection of the        target scene 72. (All as in FIGS. 7 through 9.)    -   3. Retrieve from memory a previously stored “impulse response,”        which is the pre-calculated FPA pattern produced by illumination        of the coded mask, preferably bipolar, with a plane wavefront        parallel to the mask plane, 101 and 102.    -   4. Calculate the cross-correlation of the FPA pattern with the        “impulse response,” 103. The result will be the reconstructed        image of the scene, 104.

This embodiment comprises the function of the diffracted projectioncomprising the diffracted projection; the function of the known arraypattern of the at least one original coded aperture mask comprising aprerecorded impulse response comprising a complex field produced bypassing a wavefront through the at least one original coded aperturemask onto the focal plane array; and correlating the diffractedprojection with the impulse response to produce an image of the targetscene.

The “impulse response” retrieved in the second step of the flowchart ofFIG. 10 is the calculated complex field on the FPA, rather thanintensity pattern on the FPA, with the coded-aperture mask illuminatedby a single plane wave (point source, “single star,” or “impulse”). APCT publication WO 2007/091045 by Payne uses the intensity pattern onthe detector array to determine the decoding pattern corresponding tothe coded aperture array. According to Payne, the reference object maybe a point source in which case the pattern on the detector array may beused directly as the decoding pattern or it may be used to correct atheoretical pattern for any misalignment. Unlike Payne, this inventionuses correlation of the reference complex field with the intensitypattern of the sensor, resulting in improved signal-to-noise ratio.

A fourth alternative embodiment, summarized in FIG. 11, is as follows:

-   -   1. Place an FPA behind a coded-aperture mask with a coded        pattern of apertures and obscurations—preferably, but not        limited to, URA. The feature size on the mask is preferably        small. The mask is illuminated by light arriving from the target        and/or scene.    -   2. Capture the raw diffraction pattern on the FPA, 71, resulting        in a mask-convolved, diffraction-blurred projection of the        target scene 72. (All as in FIGS. 7 through 10.)    -   3. Digitally (virtually) illuminate the resulting pattern as a        digital hologram by an array of light sources placed in the        plane of the mask, arranged in the same pattern as the        transparent areas of the complementary mask, 111. The result        will be a reconstructed set of plane wavefronts, one for each        point of the scene, for each source of the array, the plane        wavefronts related to the same point of the scene being parallel        to each other, 112.    -   4. Digitally focus and propagate to a focal plane all plane        wavefronts, and add resulting intensities of focal spots, 113.        The result will be the reconstructed image of the scene, 114.

In this embodiment, correlating occurs before holographicreconstructing; the correlating produces a hologram of the target scene;and the holographic reconstructing comprises illuminating the hologramusing an array of light sources arranged in a pattern complementary tothe at least one original coded aperture mask, producing a holographicreconstruction of the target scene; further comprising: focusing theholographic reconstruction onto an image plane, to produce an image ofthe target scene.

Four additional alternative processes are provided by modification ofthe preferred process and the first through the third alternativeprocess disclosed above in FIGS. 7 through 10, with the abovedescriptions modified to replace “calculate cross-correlation” by“correlate deconvolution.” That is, the correlating comprisingcorrelating using deconvolution. Any deconvolution method known in theart can be used, including matrix deconvolution or Fourierdeconvolution. Noise suppressions such as in Wiener deconvolution may beused.

Effective Digital Hologram

The operation of this invention may be further explained as follows.Consider exposure of the focal plane array (FPA) 14 by a wavefront froma scene and diffracted on the coded aperture mask 12, see FIG. 12 a.After correlation with the complementary mask pattern, the result of thecorrelation is identical to an intensity pattern that would be recordedon a large hologram (“correlation hologram” 121) by the interference ofan object wavefront identical to the wavefront from the scene and areference wavefront from an effective reference source 122, see FIG. 12b. The effective source is a small aperture—a single element (pixel, or“building block”) of the coded-aperture mask 12, illuminated by thewavefront from the scene. If the size of this element is sufficientlysmall, the reference wavefront 123 is simply a spherical. The distance Lfrom the source 122 to the hologram 121 is the same as the distance fromthe physical coded aperture mask 12 to the FPA 14. The size H of thehologram is comparable to the larger of the two: the coded-aperture mask12 and the FPA 14. For the case where the coded aperture mask 12 islarger than the FPA 14, the size of the hologram 121 is comparable tothe size of the mask 12, as is illustrated in FIG. 12 b when contrastedfor size with FIG. 12 a. More specifically, the hologram is effectivelyapodized, having full, maximum contrast in the center which issubstantially constant over the size of (H-h), where H is the size ofthe coded aperture mask 12, and h is the size of the FPA 14. Fartheraway from the center, the contrast of the hologram is lost and reducesto zero over the length of h, on each side of the H-h high-contrastregion of the hologram. This apodization is a direct effect of thecorrelation process.

This interpretation of the coded-aperture imaging according to thisinvention is useful for understanding the imaging process and fordetermining, among other imaging characteristics, the field of view andresolution.

The foregoing process of imaging can be considered as entailing two mainsteps: 1) physical frame capture by the FPA through correlation,equivalent to recording the large, apodized hologram with the objectwavefront arriving from the scene, mixed with the reference wavefront,i.e., diffraction on a single element of the coded aperture mask; and 2)reconstruction of the captured hologram using a similar referencewavefront.

Complementary Mask Implementations

While it is not necessary to subtract an FPA image captured with acomplementary coded aperture mask, this optional step may be beneficialin some cases, removing the effect of zero-order diffraction on a singleelement of the coded aperture mask that acts as background in theimaging according to this invention. If the optional subtraction of thecomplementary pattern is desired, it may be implemented as follows. Inthe following, several implementations for capturing diffractionpatterns from original and complementary coded-aperture masks aredisclosed. It is understood that possible implementations of capturingsuch diffraction patterns are not limited to the specific examplespresented in the following, and other implementations that will becomeobvious to a person having ordinary skill in the art may similarly beused in this invention. These obvious variants include, but are notlimited to, transmission-type masks, reflective masks, and phase masks.

Polarization Masks

As illustrated in FIGS. 23 and 24, a polarization coded-aperture mask 12may be used, with two groups of polarizers. Within each group,polarizers are oriented in parallel; between the two groups, thepolarizer orientations are at 90 degrees. The groups preferably coverequal total surface areas on the mask. For example, they may follow thepattern of a uniformly redundant array (URA), in which zero values wouldbe represented by zero-angle polarizer orientation, and non-zero valuesby the group of polarizers at 90 degrees to the first group.

As in FIG. 23, the diffraction patterns produced by the polarizationmask can be separated into the “original” and “complementary” using anumber of methods obvious to a person skilled in the art. For example,the two optical patterns may be separated by a polarization beamsplitter231 into two optical paths, each bringing light to one of two identical,co-registered FPA 14 Alternatively, as in FIG. 24, one optical path canbe used, with a fast electronically controlled polarization rotator (PR)241, such a ferroelectric liquid crystal (FLC) PR, combined with ananalyzer behind the PR. Two consecutive frames may be captured by thesame FPA 14, with the PR 241 immediately preceding the FPA 14 in theoptical path, and the state of the PR 241 changing between the frames.

In sum, this implementation comprises a polarizing coded aperture maskcomprising both original mask polarizers with a first polarizationorientation and complementary mask polarizers with a second polarizationorientation; wherein: diffracted projections produced by the originalmask polarizers and diffracted projections produced by the complementarymask polarizers are respectively separated into the diffractedprojection of the target scene and a complementary diffracted projectionof the target scene.

Several other complementary mask implementations following on the basicconcepts illustrated in FIGS. 23 and 24 are further reviewed below.

Diffraction Masks

The elements of the coded-aperture mask may be miniature diffractiongratings. The two states of the coded-aperture mask may be representedby two distinct directions of the gratings rulings, e.g., at 90 degreesto each other. At diffraction angles other than zero, the light from thetwo groups of gratings will be separated on the FPA, to be used in theprocessing of the holograms.

This implementation comprises a diffraction grating coded aperture maskcomprising both original mask grating rulings and complementary maskgrating; wherein: diffracted projections produced by the original maskgrating rulings and diffracted projections produced by the complementarymask grating rulings are respectively separated into the diffractedprojection of the target scene and a complementary diffracted projectionof the target scene.

Refractive Masks

The two groups of the coded aperture mask elements may be refractiveelements, such as prisms, to separate the two diffractive patterns bydirection of propagation, similar to diffraction masks.

Here, we have a refractive coded aperture mask comprising both originalmask refractive elements and complementary mask refractive elements;wherein: diffracted projections produced by the original mask refractiveelements and diffracted projections produced by the complementary maskrefractive elements are respectively separated into the diffractedprojection of the target scene and a complementary diffracted projectionof the target scene.

Reflective Masks

The two groups of elements of the coded-aperture mask may be transparentand reflective elements, e.g., produced by a patterned mirror on atransparent substrate.

In this implementation, one has a mixed transparent and reflective codedaperture mask comprising both transparent and reflective elements;wherein: diffracted projections produced by the transparent elements anddiffracted projections produced by the reflective elements are separatedinto the diffracted projection of the target scene and a complementarydiffracted projection of the target scene.

Dynamic Masks

Here, the coded-aperture mask is dynamic, e.g., implemented as aliquid-crystal display, or a MEMS device. The “normal” and thecomplementary masks are formed in sequence, and two FPA frames arecaptured, one with each mask. The two frames are subtracted in theprocessing.

This implementation employs a dynamic coded aperture mask for forming insequence, both the at least one original coded aperture mask and thecomplement of the at least one original coded aperture mask; wherein:diffracted projections produced by the at least one original codedaperture mask and diffracted projections produced by the complement ofthe at least one original coded aperture mask are respectively separatedinto the diffracted projection of the target scene and a complementarydiffracted projection of the target scene.

Performance Factors

Twin Images

Referring to FIG. 13, each “holographic lens” is in fact an on-axishologram similar to a Gabor in-line Fresnel hologram. The second andthird terms in (3) represent two “twin” images that are present afterreconstruction by the spherical wave. One image is at infinity, formedby the reconstructed plane wavefront 34. The other image is a virtualimage, formed by a diverging spherical wave. Physically, the two imagesare created by diffraction on the concentric near-circular rings of thehologram. The two images are the two diffractive orders (plus first andminus first) of the diffractive structure of the hologram. Higher ordersare not present due to the smooth, sinusoidal profile of the hologramdensity (pixel values).

The twin spherical wavefront 131 forms a virtual image of thepoint—center of the reconstructing spherical wavefront 132—at half thedistance between the reference point source 122 and the hologram 24. Theradius of curvature of the twin wavefront is half the curvature of thereconstructing wavefront.

Either of the two images can be used in digital processing to arrive atthe imagery of the scene. The unused twin image will create a defocused,wide-spread background, which can be removed by digital filtering.

Spatial Coherence

Under natural illumination, different resolved points of the scene areincoherent with each other. The reason for this, as is know fromprinciples of physical optics, is that diffraction limited resolution is

${1.22 \cdot \frac{\lambda}{D}},$whereas angular size of spatial coherence is

$0.16 \cdot {\frac{\lambda}{D}.}$As a result, diffraction patterns of different points of the scene addas intensities on the FPA (stacked holograms from each point of thescene, independent from each other). At reconstruction, each pointproduces an independent wavefront (plane wavefront if the point is atinfinity, spherical if the point is close). Jointly, the overallwavefront from the scene is faithfully reconstructed.

Resolution

In diffraction-free coded-aperture imaging according to the prior art,resolution is defined by the angular size of the aperture element asviewed from the FPA. In the presence of diffraction, the practicalresolution is considered in the prior art to be even worse, with thediffraction also limiting the minimal feasible aperture size.

In contrast to the limitations ordinarily understood to be inherent inusing coded aperture imaging for larger wavelengths, the imagingresolution according to this invention is not directly defined by thefeature size of the mask. Rather, the imaging resolution of thisinvention is similar to the resolution of a hologram. It is ultimatelydiffraction-limited by the effective size of the hologram and by thewavelength.

The hologram effective size is defined by four characteristic, orcritical, sizes: The first is the size of the spot on the FPA fromdiffraction on the smallest feature (“building block”) of the mask. Thisspot size is a product of the diffraction angle on the mask featuremultiplied by the mask-to-FPA distance. The second critical size is thatof the “holographic lens” limited by the pixel resolution of the FPA.The third critical size is that of the “holographic lens” limited by thecoherence length (spectral width) of the light from the point of thescene or target. The hologram of this invention is a result ofcorrelation of the diffraction pattern on the FPA with the pattern ofthe (complementary) coded-aperture mask. The fourth critical size is thesize of this correlation. If the size of the “holographic lens,” limitedby the first three critical sizes, exceeds the size of the hologram(correlation output), the smaller size—the hologram in this case—willlimit the imaging resolution. Limiting the imaging resolution will bethe compound effect (convolution) of the four critical sizes. Theresolution will be no better than defined by the smallest of the foursizes.

The size of the “holographic lens” is limited by pixel pitch of the FPAand by the coherence length of the source. It is a reasonable tradeoffto choose the spectral bandwidth for a given FPA pixel pitch so that thetwo sizes are close. Filtering to a narrower spectral band will causelight loss without the payoff of higher resolution. Wider spectral widthwill leave pixels on the FPA underused. For example, at the 4 umwavelength, distance from the mask to the FPA of 100 mm, pixel pitch 25um, and on-axis target, the pixel-limited “holographic lens” diameter is32 mm. The corresponding angular resolution is 3 milliradians. The“lens” diameter increases linearly with the mask-FPA distance L. Thetradeoff spectral bandwidth is proportional to 1/L.

If the target has narrow-band spectral features of interest on a broadspectral background, the narrow-band features may produce larger“holographic lens” instances, and the related high spatial frequenciesin the reconstructed image. In this case, broader filter bandwidth maybe chosen, compared to the above tradeoff.

As long as the mask element is sufficiently small, and the diffractionby the mask produces a spot larger than the size of the “holographiclens” otherwise defined (by the FPA pixel pitch and by the coherencelength), the coded aperture mask does not limit the resolution or fieldof view of the imaging. A smaller element size of the coded aperturemask also makes the reconstructing wavefront match a spherical wavefrontmore closely, which may simplify the digital hologram reconstruction.

When the “holographic lens” is small compared to the “correlationhologram”, and a limited field of view is of interest, much of thecoded-aperture mask may not contribute to the diffraction pattern thatproduces the portions of the hologram related to the field of view ofinterest. In this case, an adaptive coded aperture may be useful, whichwould close the unused apertures, to reduce background illumination ofthe FPA.

Field of View

In conventional digital holography, the resolution of the imaging sensorlimits the field of view, for two reasons. First, once the angle betweenthe object wavefront and the reference wavefront exceeds a certainthreshold value, the size of the interference fringes falls below thepixel pitch of the FPA, and the hologram becomes undersampled. Second,at reconstruction, the periodic pixel structure of the FPA acts as adiffraction grating, producing higher diffraction orders. The higherorders produce repetitive, overlapping instances of the image. To avoidthis, the field of view needs to be limited.

Unlike conventional digital holography, this invention provides for eachpoint of the scene its own reference beam, so the digital reconstruction(or optical reconstruction, with SLM displaying the hologram,illuminated by a laser beam) is possible for all look angles. The tiltof the object wavefront (look angle) determines the center location ofthe “holographic lens” 24 on the FPA 14. The FPA pixel size affects themaximum diameter of the “holographic lens” 24 for look angle, related todiffraction-limited resolution of the reconstructed wavefront. For lookangles within the angular size of the mask as viewed from the FPA 14, nolimitation on the look angle and the field of view is posed in theimaging, according to this invention.

Provided sufficient coherence length (e.g., filtered light, or laserillumination), it is possible that the size of the “holographic lens” 24exceeds the size of the FPA 14. At extreme look angles, the center ofthe “holographic lens” 24 may be outside the “correlation hologram” 121with only the periphery of the “holographic lens” 24 inside “correlationhologram” 121. The condition when the frequency of the fringes reachesthe Nyquist criterion dictated by the FPA pixel pitch defines theextreme look angle of the system. In principle, the extreme look angleand FOV may exceed the geometrical angle at which the mask 12 is viewedfrom the FPA 14, including the situation when the light from the scenearrives at the FPA as the “signal wave” outside the boundary of thecoded-aperture mask.

Enhancement of Resolution and Field of View

In a conventional lens camera, resolution and field of view are usuallymutually limiting: providing wider field of view reduces resolution, andvice versa.

In the imaging according to this invention, the field of view is definedby the angular size of the coded aperture mask 12 as viewed from the FPA14. The resolution is defined by the effective size of the “correlationhologram” 121, the result of correlation of the capture FPA frame withthe coded-aperture mask pattern, see FIG. 12. The hologram effectivesize (H-h) may exceed the size of the FPA 14 (h); in principle, it mayreach and slightly exceed the size of the coded aperture mask 12. (“Inprinciple,” because the limitation exists related to the FPA pixelresolution, fringe spatial frequency, and hence the size of individual“holographic lenses” for each point of the scene.) This means that alarger coded-aperture mask 12 size H may simultaneously translate into alarger field of view and a higher resolution.

FIGS. 14 through 16 illustrate envisioned approaches to enhancingresolution and field of view of the imaging system according to thisinvention.

FIG. 14 illustrates interference of a scattered spherical wavefront 142with light from the scene 11, and how frequency of the fringes near theedge 141 of FPA 14 may reach the pixel spatial frequency. A second focalplane array 14 is tilted so that fringes beyond the former (untilted)FPA 14 are still resolved. This approach, wherein at least one originalcoded aperture mask and the focal plane array are substantially notparallel to one another, allows one to overcome the limitation on theuseful size of the FPA limited by its pixel resolution.

FIG. 15 illustrates the use of an optical element (e.g., lens) 151 forconcentrating the light energy on the FPA 14. For large coded-aperturemasks 12, diffracted light from off-axis portions may miss the FPA andthus not contribute to producing the “holographic lens,” which in mostembodiments is the output of the correlation processing. In such cases,a lens 151 may be used as shown in FIG. 15 to bend the rays and to bringdiffracted light to the FPA 14. The lens does not produce a focusedimage; its function is limited to directing/concentrating more of thediffracted light on the FPA 14 to reduce this wavefront loss.

It is to be understood that although most of the preceding discussionassumed a planar coded-aperture mask parallel to a planar FPA, codedaperture masks non-parallel to the FPA, and non-planar coded aperturemasks and non-planar FPAs, as well as multiple FPAs, may be used in thisinvention. FIG. 16 illustrates how a curved, non-planar, coded-aperturemask 12 in combination with multiple FPA or with a curved FPA 14 offersextended field of view coverage.

Advanced Capabilities

Imaging in Non-Monochromatic Light

The effect of finite coherence length, defined by the spectral width ofthe imaged point of the scene, was already mentioned in relation to thesize of the “holographic lens” 24 and imaging resolution. The broaderthe spectral bandwidth, the shorter is the coherence length, the smalleris the “holographic lens” diameter, and so the worse is the imagingresolution.

The above statement applies to smooth, “well-behaved” spectra. In somecases, the broad spectrum may contain fine spectral features, such asnarrow peaks or troughs, e.g., from the target being “painted” by alaser beam, or having narrow emission or absorption spectral lines, suchas in missile plumes. In these cases, the broad and smooth spectralcomponents will result in smaller “holographic lenses,” producing atarget image at a lower spatial resolution; however, additive to thesewill be larger “holographic lenses” from the narrow spectral features,producing image features at higher spatial resolution.

Narrowing the imaging spectral range with bandpass or notch filters willincrease spatial resolution as well. In this invention, higher spectralresolution results in higher spatial resolution—the two resolutions arenot mutually competitive, unlike in some other imaging systems such ashyperspectral images where a limited number of pixels are shared betweena number of “spectral planes.” No such tradeoff is required in thisinvention.

Hyperspectral Imaging

The process of digital hologram reconstruction is dependent on thewavelength used in recording the hologram. Using a “wrong” wavelengthvalue during reconstruction will result in a defocused image. Ifdifferent wavelengths are present during recording in the arrivingwavefront (as may be the case in many practical situations), whenreconstruction is done using one specific wavelength, the one spectralcomponent of the recording whose wavelength is used at reconstruction,will be in focus. At the same time, other spectral components of theimage will be defocused. Multiple digital reconstructions at manywavelengths will result in many instances of the image, with differentspectral components in focus. The multiple digital reconstructions maybe combined with additional spatial filtering of the image, to arrive ata hyperspectral image of the scene (spectral cube).

Change Detection

During recording, each point of the distant scene arrives at theaperture or aperture array as a plane wave. It is registered on the FPAas a “holographic lens”: a Fresnel hologram, with the referencespherical wave that has zero phase shift with the plane wave along thedirection of propagation of the plane wave. At reconstruction, all ofthe “holographic lenses,” each respective to a point of the target sceneand an arriving plane wavefront, are illuminated by the same coherentspherical wavefront. As a result, the whole reconstructed complexwavefront (comprising all component plane wavefronts) is coherent, eventhough the original scene was spatially incoherent.

This makes it possible to process the image of the scene by means ofcoherent optics (or equivalent digital/virtual processing), as if thescene was displayed on a transparency and illuminated by a laser beam. Awealth of processing algorithms of coherent optics is thereforeapplicable to extracting information from the scene, including change intime (captured in a temporal series of frames by the FPA).

One such processing algorithm is change detection. Many algorithms forchange detection are described in the literature, and all may besuitably applied here within the capacities of persons of ordinary skill

Referring to FIG. 17, a possible implementation of change detectionaccording to the present invention is illustrated by way of example andnot limitation in the following:

Consider image h(x,y) being a reference image of a scene; and imageh′(x, y) is a shifted image of the same scene, containing a changeg=h′−h, or, more expansively:h′(x,y)=h(x−a,y−b)+g(x,y)

The change can be detected by using two Fourier-domain filters and anonlinear filter.

The first Fourier filter is

${F_{1}\left( {p,q} \right)} = \frac{1}{{H\left( {p,q} \right)} + 1}$where H(p,q) is the Fourier transform of the function h(x, y). Thesecond filter is F₂(p,q)=H(p,q)

A nonlinear filter (converter), N(·), is applied to the image-planeintensity distribution, enhancing the image contrast. It may beimplemented, for example not limitation, as

${N(t)} = \left| \;\begin{matrix}{{{2 \cdot \left( {t - \frac{\max}{2}} \right)}\mspace{14mu}{if}\mspace{14mu} t} > \frac{\max}{2}} \\{0\mspace{14mu}{otherwise}}\end{matrix} \right.$

Thus, starting with h′(x,y) and applying F₁(p,q) followed by N(·)followed by F₂(p,q), and then subtracting off the shifted h′(x,y) yieldsg(x,y), which measures the difference h′−h.

An experimental result of numeric simulation of this process is shown inFIG. 18, where the base image contains letters “ASI” and the changedimage contains the same letters, shifted, and also letters “Inc.” Theprocessing result is the dimmed letters “ASI” and the bright letters“Inc.,” indicating the change.

3D Imaging

If the object is at a finite distance (close enough for the incomingwavefront from the object being recorded to have a spherical aspect)from the “camera” (from the coded-aperture mask), the proposed imagingwill capture the 3D structure of the object. Different views of theobject from different viewpoints will be holographically captured andreconstructed.

Unlike conventional holography, no local reference beam is required; thereference wavefront for every point of the object is produced from thesame wavefront, bearing certain similarity to the Smarttpoint-diffraction interferometer.

Additional capabilities of this invention for 3D imaging may berealized, as disclosed earlier, by placing the coded-aperture mask 12and the FPA 14 at a non-zero angle, so that they are not parallel toeach other, as illustrated by FIGS. 14 and 16.

Coherent Aperture Synthesis

With laser illumination, even at long, e.g., ˜100 km ranges, the presentinvention may be used to capture the scene and target wavefront,including phase. Referring to FIG. 16, multiple imagers of thisinvention may produce complex, amplitude-and-phase images of the scene,which may be coherently combined to arrive at a higher imagingresolution, potentially diffraction-limited by the composite aperturesof the multiple coded-aperture arrays. The aperture arrays may takeshapes other than planar, including conformal aerodynamic shapes, orother conformal shapes. A possible embodiment may include multipleconformal coded-aperture arrays installed on the surface of the wing orbody of a flying platform with one or multiple FPA installed inside theplatform.

In other words, with coherent laser illumination, effective aperture canbe large, synthesized from multiple individual coded-aperture arrays,with the resolution diffracted-limited by the composite synthesizedaperture.

Polarimetric Imaging

Polarization-sensitive, or polarimetric, imaging is provided by thisinvention, with a coded-aperture mask comprising four sorts of surfaceareas (elements, or pixels, or “pinholes”): 1) opaque elements; 2)transparent elements of a first kind, covered with a polarizer materialin a first orientation; 3) transparent elements of a second kind,covered with a polarizer material in a second orientation, perpendicularto the first orientation; and 4) transparent elements of a third kind,transparent to either polarization. The patterns formed by thetransparent elements of the first, second, and third kinds arestatistically independent, with zero or nearly zero cross-correlation.

This is illustrated in FIG. 25. The overall polarization coded-aperturemask 12 is shown on the left. A magnified fragment of the mask is shownon the right, with the four types of elements: transparent elements 251as shown in white; opaque elements 252 shown in black; transparentelements of a first kind 253 indicated by vertical patterns, comprising(e.g., covered with) a polarizer material in the first orientation; andtransparent elements of a second kind 254 indicated by horizontalpatterns, comprising (e.g., covered with) a polarizer material in asecond orientation, perpendicular to the first orientation.

The pattern formed on the FPA 14 by the light form the scene afterpassing through the mask is processed using any of the processingalgorithms according to this invention. Processing with the mask patternof the first kind results in a polarization image with the polarizationcomponent along the first orientation. Processing with the mask patternof the second kind results in a polarization image with the polarizationcomponent along the second orientation, perpendicular to the firstorientation. As a result, polarimetric imaging is provided, withorthogonal polarization components measured and presented in therespective pixel values of the two output images, for every point of thescene and/or target.

The diffracted projections from this mask are processed into arepresentation of the target scene in the first and second polarizationorientations by correlating a function of the diffracted projection witha function of the pattern of the transparent elements, respectively, inthe first and second polarization orientations, and for eachpolarization orientation, by using at least one reconstructing wavefrontfor holographic reconstructing

Correction of Aberrations

One or more optical elements or components (system optics) may precedethe coded aperture mask in the optical path. Such an optical componentmay be, for example but with no limitation, a protective window or dome.This preceding component may introduce aberration in the opticalwavefront arriving from the scene. The digitally reconstructedwavefront, produced by the processing algorithms according to variousembodiments of this invention, includes both the phase of theun-aberrated wavefront form the scene and the phase aberrations of thepreceding optics. The aberrations of the optics can be subtracted fromthe reconstructed phase. The corrected digital wavefront can then be“digitally focused” to produce an aberration-free image.

Alternatively, the digital pattern of the coded-aperture mask can bedistorted (remapped) prior to use in the reconstruction algorithm ofthis invention. Proper distortion, or remapping, of the mask patternwill produce aberrations in the reconstructed wavefront that willcompensate aberrations of the preceding optics.

If the aberrations of the preceding optics are known, e.g., from theoptical prescription of the optics or from an independent measurement,the aberrations of the optics are subtracted from the reconstructedwavefront, to produce aberration-free wavefront and image. If theaberrations of the optics are unknown, they can be determined in asimple calibration procedure as follows: The coded-aperture imagingsensor with image processing according to this invention, assembled withthe protective window or other optics, is illuminated by a calibrationplane wavefront of very low aberrations. The phase of the digitalwavefront reconstructed according to this invention is the aberration ofthe protective window or other optics. The measured optics aberration isstored in memory or in a computer storage device to be subtracted infuture processing, to produce the aberration-free wavefront and image.The calibration procedure may be repeated for different orientations, ordirections of incidence, of the calibration plane wavefront. Othershapes of the calibration wavefront, including but not limited tospherical, are equally applicable for this calibration.

Compensation of Atmospheric/Environmental Turbulence

Atmospheric, i.e., environmental turbulence introduces aberrations intoa wavefront. If the aperture size exceeds the Fried parameter R0, whichis the characteristic size of the aberration due to turbulence, imagequality is degraded. Similar to correction of aberrations of protectivewindows or domes, the aberration due to the atmospheric/environmentalturbulence can be removed from the digital wavefront reconstruction byprocessing according to this invention. Unlike the static aberrationjust discussed of the protective or other optics, aberrations fromatmospheric turbulence are dynamic, changing from one captured FPA frameto another.

The atmospheric aberrations are determined and removed by maximizing a“sharpness function” of the resulting image, or other means known in theart, e.g., similar to the methods used in adaptive optics. Unlikeadaptive optics, compensation of atmospheric/environmental turbulenceaccording to this invention is performed digitally, i.e. virtually,i.e., computationally, either in real time or in post-processing. Inreferring to atmospheric and/or environmental turbulence, no limitationto the earth's atmosphere and/or environment is intended. Rather,recognizing that obstructions and distortions and aberrations can occuralong the light wave propagation path in a wide variety ofcircumstances, this is intended to refer to the “atmosphere/environment”between the target scene and the coded aperture imaging system of thisdisclosure, from whatever source or origin, and for whatever reason.

Alternatively, the following method provides for correction ofatmospheric/environmental turbulence: The reconstructed wavefront issubdivided, in the digital processing, into multiple coherentsubapertures (“sub-wavefronts”), each smaller than Fried's parameter,R0. Each sub-wavefront is digitally focused to produce an image of thedistant object. The resolution of each image is low, diffraction-limitedby the size of the subaperture. Correlation of the multiple images iscalculated, to determine the shift of image and the related tilt of therespective sub-wavefront, caused by the atmospheric turbulence. Theestablished tilts are removed from each reconstructed sub-wavefront. Allcorrected sub-wavefronts are coherently added, to produce the correctedwavefront over the entire aperture of the telescope, with the effect ofthe turbulence removed. The corrected wavefront is digitally focused, toproduce the high-resolution image, diffraction-limited by the overalltelescope aperture.

The same method can be used to remove static wavefront aberrations, withthe size of the subapertures smaller than the characteristic size of theaberration, i.e., the subaperture size over which variation of the phaseaberrations is sufficiently small. The aberration is sufficiently smallif the related phase ramp causes image displacement smaller than thediffraction-limited spot size, defined by the overall aperture (not thesubaperture).

Numerical modeling of coded aperture imaging according to this inventionis illustrated by FIG. 19. Experimental implementation demonstration ofcoded aperture imaging according to this invention is illustrated byFIGS. 20-22. FIGS. 20 and 21 show experimental results with subtraction(earlier steps 2 a and 2 b) of two FPA frames captured withcomplementary coded-aperture masks, formed on a liquid-crystal microdisplay. FIG. 22 shows single-frame reconstructions, performed from asingle FPA frame, without this optional step of subtracting the framescaptured with complementary coded-aperture masks.

It is important to reemphasize that throughout this disclosure, theprocessing of the diffracted projection of the target scene passedthrough the coded aperture can employ a variety of physical opticaland/or virtual digital techniques and methods. Thus, when thisdisclosure and its associated claims make reference, for example notlimitation, to using a reconstructing wavefront for holographicreconstructing, it is understood that the reconstructing wavefront canbe a real physical wavefront, or a virtual wavefront which iseffectively a digital simulation. Thus, for another example withoutlimitation, when this disclosure and its associated claims speak ofpassing a wavefront through a complement of the coded aperture mask, itis understood that this complement may be a physical mask, and/or it maybe a virtual mask which is digitally represented/simulated. And, it isunderstood that the passage of light through this mask may similarly bea real, physical passage of light through a real, physical mask, or thatthis may all take place virtually, by computerized digital simulationand processing. For another example without limitation, when referenceis made to “focusing” light, it is understood that this may beimplemented with physical lenses or similar optical elements, and/orwith digital representations of lenses/optical elements and of thepassage of light through these lenses/optical elements. More broadly, itis again to be emphasized that for many of the processing stepsdisclosed and claimed herein, the implementation of that step may bereal and physical, and/or it may be virtual and digitally simulated, andthat there is no limitation to be inferred to one or the other form ofimplementation unless such a limitation is explicitly stated or recited.The only step which is always a real, physical step, is the passing oflight from the target scene through the coded aperture mask onto thefocal plane, for it is this physical image that all of the remainingprocessing steps are motivated to reconstruct, via a variety ofprocesses which may be physical/optical, digital/virtual, or both.

While the foregoing written description of the invention enables one ofordinary skill to make and use what is considered presently to be thebest mode of this invention in various embodiment, those of ordinaryskill will understand and appreciate the existence of variations,combinations, and equivalents of the specific embodiment, method, andexamples herein. The invention should therefore not be limited by theabove described embodiments, methods, and examples, but should encompassall embodiments and methods within the scope and spirit of the inventionas claimed. Thus, while only certain preferred features of the inventionhave been illustrated and described, many modifications, changes andsubstitutions will occur to those skilled in the art. It is, therefore,to be understood that the appended claims are intended to cover all suchmodifications and changes as fall within the true spirit of theinvention.

1. A method for coded aperture sensing, comprising: passing at least onescene wavefront from a target scene through at least one original binarycoded aperture mask comprising transparent and opaque portions thereof,onto a focal plane array, producing a diffracted projection of thetarget scene; and processing said diffracted projection into arepresentation of the target scene by correlating a function of saiddiffracted projection with a function of a known array pattern of saidat least one original binary coded aperture mask and by using at leastone reconstructing wavefront for holographic reconstructing.
 2. Themethod of claim 1, said function of said known array pattern of said atleast one original binary coded aperture mask comprising a complement ofsaid at least one original binary coded aperture mask.
 3. The method ofclaim 1, wherein: said correlating occurs before said holographicreconstructing; said correlating produces a hologram of the targetscene; said holographic reconstructing comprises Fresnel illuminatingsaid hologram into a holographic reconstruction of the target scene. 4.The method of claim 3, further comprising focusing said holographicreconstruction onto an image plane, producing an image of the targetscene.
 5. The method of claim 1, further comprising: passing at leastone scene wavefront from a target scene through a complement of said atleast one original binary coded aperture mask, producing a complementarydiffracted projection of the target scene; wherein: said function ofsaid diffracted projection comprises a difference between saiddiffracted projection, and said complementary diffracted projection. 6.The method of claim 1, wherein: said correlating occurs before saidholographic reconstructing; said correlating produces a hologram of thetarget scene; and said holographic reconstructing comprises illuminatingsaid hologram into a holographic reconstruction of said target scene;further comprising: propagating said holographic reconstruction onto animage plane and calculating intensity at points of said image plane,producing an image of the target scene.
 7. The method of claim 1, saidfunction of said diffracted projection comprising a portion of saiddiffracted projection.
 8. The method of claim 1, said function of saidknown array pattern of said at least one original binary coded aperturemask comprising a morphed pattern of said at least one original codedaperture mask.
 9. The method of claim 1, wherein: said holographicreconstructing occurs before said correlating; said holographicreconstructing comprises illuminating said diffracted projection of thetarget scene as a hologram of said diffracted projection to produce apattern image; and said correlating comprises correlating said patternimage of said hologram with said function of said known array pattern ofsaid at least one original binary coded aperture mask, producing animage of the target scene.
 10. The method of claim 1, furthercomprising: said function of said diffracted projection comprising saiddiffracted projection; said function of said known array pattern of saidat least one original binary coded aperture mask comprising aprerecorded impulse response comprising a complex field produced bypassing a wavefront through said at least one original binary codedaperture mask onto said focal plane array; and correlating saiddiffracted projection with said impulse response to produce an image ofthe target scene.
 11. The method of claim 1, wherein: said correlatingoccurs before said holographic reconstructing; said correlating producesa hologram of the target scene; and said holographic reconstructingcomprises illuminating said hologram using an array of light sourcesarranged in a pattern complementary to said at least one original binarycoded aperture mask, to produce a holographic reconstruction of thetarget scene; further comprising: focusing said holographicreconstruction onto an image plane, produces an image of the targetscene.
 12. The method of claim 3, said correlating comprisingcorrelating using deconvolution.
 13. The method of claim 6, saidcorrelating comprising correlating using deconvolution.
 14. The methodof claim 9, said correlating comprising correlating using deconvolution.15. The method of claim 10, said correlating comprising correlatingusing deconvolution.
 16. The method of claim 2, further comprising: apolarizing coded aperture mask comprising both original mask polarizerswith a first polarization orientation and complementary mask polarizerswith a second polarization orientation; wherein: diffracted projectionsproduced by said original mask polarizers and diffracted projectionsproduced by said complementary mask polarizers are respectivelyseparated into said diffracted projection of the target scene and acomplementary diffracted projection of the target scene.
 17. The methodof claim 2, further comprising: a diffraction grating coded aperturemask comprising both original mask grating rulings and complementarymask grating; wherein: diffracted projections produced by said originalmask grating rulings and diffracted projections produced by saidcomplementary mask grating rulings are respectively separated into saiddiffracted projection of the target scene and a complementary diffractedprojection of the target scene.
 18. The method of claim 2, furthercomprising: a refractive coded aperture mask comprising both originalmask refractive elements and complementary mask refractive elements;wherein: diffracted projections produced by said original maskrefractive elements and diffracted projections produced by saidcomplementary mask refractive elements are respectively separated intosaid diffracted projection of the target scene and a complementarydiffracted projection of the target scene.
 19. The method of claim 2,further comprising: a mixed transparent and reflective coded aperturemask comprising both transparent and reflective elements; wherein:diffracted projections produced by said transparent elements anddiffracted projections produced by said reflective elements areseparated into said diffracted projection of the target scene and acomplementary diffracted projection of the target scene.
 20. The methodof claim 2, further comprising: a dynamic coded aperture mask forforming in sequence, both said at least one original binary codedaperture mask and said complement of said at least one original binarycoded aperture mask; wherein: diffracted projections produced by said atleast one original binary coded aperture mask and diffracted projectionsproduced by said complement of said at least one original binary codedaperture mask are respectively separated into said diffracted projectionof the target scene and a complementary diffracted projection of thetarget scene.
 21. The method of claim 1, said using at least onereconstructing wavefront for holographic reconstructing comprisingincreasing spatial resolution by narrowing a spectral range with afilter selected from the filter group consisting of: bandpass filtersand notch filters.
 22. The method of claim 1, said using at least onereconstructing wavefront for holographic reconstructing comprising:using a plurality of reconstructing wavefronts of different wavelengthsto focus different spectral components of the holographicreconstruction; and combining said different spectral components of saidholographic reconstruction, producing a hyperspectral image of thetarget scene.
 23. The method of claim 1, said using at least onereconstructing wavefront for holographic reconstructing comprisingapplying a change detection algorithm to said representation of thetarget scene.
 24. The method of claim 23, wherein said representation ofthe target scene is denoted by h(x,y), said change detection algorithmcomprising: producing a shifted image of said representation of thetarget scene denoted by h′(x, y), such that:h′(x,y)=h(x−a,y−b)+g(x,y), where g(x,y) denotes a difference betweensaid representation of the target scene and said shifted image of saidrepresentation of the target scene; detecting said difference g(x,y),using a first Fourier filter${{F_{1}\left( {p,q} \right)} = \frac{1}{{H\left( {p,q} \right)} + 1}},$where H(p,q) is the Fourier transform of the function h(x, y), using asecond Fourier filterF ₂(p,q)=H(p,q), and using a nonlinear filter N(·); by: starting withh′(x,y) and applying F₁(p,q) followed by N(·) followed by F₂(p,q), andthen subtracting off the shifted h′(x,y) to yield g(x,y).
 25. The methodof claim 1, wherein said at least one original binary coded aperturemask and said focal plane array are substantially parallel to oneanother.
 26. The method of claim 1, wherein said at least one originalbinary coded aperture mask and said focal plane array are substantiallynot parallel to one another.
 27. The method of claim 1, furthercomprising at least one optical element for concentrating said at leastone scene wavefront onto said focal plane array to reduce wavefrontloss.
 28. The method of claim 1, wherein said at least one originalbinary coded aperture mask is substantially planar.
 29. The method ofclaim 1, wherein said at least one original binary coded aperture maskis substantially non-planar.
 30. The method of claim 29, wherein a shapeof said at least one original binary coded aperture mask is conformal.31. The method of claim 2, further comprising: a polarimetric codedaperture mask comprising all four of: opaque elements; transparentelements polarized in a first polarization orientation; transparentelements polarized in a second polarization orientation; and unpolarizedtransparent elements transparent to said first polarization orientationand said second polarization orientation; wherein said transparentelements and said opaque elements are arranged in statisticallyindependent patterns; and processing said diffracted projection into arepresentation of the target scene in said first and second polarizationorientations by correlating a function of said diffracted projectionwith a function of the pattern of said transparent elements,respectively, in said first and second polarization orientations, andfor each polarization orientation, by using at least one reconstructingwavefront for holographic reconstructing.
 32. The method of claim 1,further comprising compensating for known aberrations introduced bysystem optics used between the target scene and said original binarycoded aperture mask, by subtracting said known aberrations from theholographic reconstruction, to produce an aberration-compensatedholographic reconstruction.
 33. The method of claim 1, furthercomprising compensating for known aberrations introduced by systemoptics used between the target scene and said original binary codedaperture mask, by: remapping a digital pattern of said original binarycoded aperture mask, as a function of said known aberrations; and usingthe remapped digital pattern for said holographic reconstructing, toproduce an aberration-compensated holographic reconstruction.
 34. Themethod of claim 1, further comprising compensating for unknownaberrations introduced by system optics used between the target sceneand said original binary coded aperture mask, by a calibration procedurecomprising: passing at least one calibration wavefront through saidsystem optics in combination with said at least one original binarycoded aperture mask onto the focal plane array, producing an aberrateddiffracted projection of a source of said calibration wavefront;deducing a measured system optics aberration from said aberrateddiffracted projection; and subtracting said measured system opticsaberration during processing of said diffracted projection of the targetscene, to produce an aberration-compensated holographic reconstruction.35. The method of claim 1, further comprising compensating for unknownatmospheric aberrations introduced between the target scene and saidoriginal binary coded aperture mask, by computationally maximizing asharpness function during processing of said diffracted projection ofthe target scene, to produce an aberration-compensated holographicreconstruction.
 36. The method of claim 1, further comprisingcompensating for unknown atmospheric aberrations introduced between thetarget scene and said original binary coded aperture mask, by:subdividing at least one wavefront of the holographic reconstruction,into a plurality of coherent sub-wavefronts, each smaller than Fried'sparameter; computationally refocusing each sub-wavefront to producecorresponding object images; correlating said object images to calculateshifts and tilts of the corresponding sub-wavefronts, caused by theatmospheric turbulence; removing the calculated shifts and tilts toproduce a corresponding plurality of corrected sub-wavefronts; andcoherently combining said corrected sub-wavefronts, to produce anaberration-compensated holographic reconstruction.
 37. A system forcoded aperture sensing, comprising requisite optical elements,computerized storage and computerized processing for: passing at leastone scene wavefront from a target scene through at least one originalbinary coded aperture mask comprising transparent and opaque portionsthereof, onto a focal plane array, producing a diffracted projection ofthe target scene; and processing said diffracted projection into arepresentation of the target scene by correlating a function of saiddiffracted projection with a function of a known array pattern of saidat least one original binary coded aperture mask and by using at leastone reconstructing wavefront for holographic reconstructing.
 38. Thesystem of claim 37, said function of said known array pattern of said atleast one original binary coded aperture mask comprising a complement ofsaid at least one original binary coded aperture mask.
 39. The system ofclaim 37, wherein: said correlating occurs before said holographicreconstructing; said correlating produces a hologram of the targetscene; said holographic reconstructing comprises Fresnel illuminatingsaid hologram into a holographic reconstruction of the target scene. 40.The system of claim 39, further comprising said requisite opticalelements, computerized storage and computerized processing for focusingsaid holographic reconstruction onto an image plane, producing an imageof the target scene.
 41. The system of claim 37, further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for: passing at least one scene wavefront from a target scenethrough a complement of said at least one original binary coded aperturemask, producing a complementary diffracted projection of the targetscene; wherein: said function of said diffracted projection comprises adifference between said diffracted projection, and said complementarydiffracted projection.
 42. The system of claim 37, wherein: saidcorrelating occurs before said holographic reconstructing; saidcorrelating produces a hologram of the target scene; and saidholographic reconstructing comprises illuminating said hologram into aholographic reconstruction of said target scene; further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for: propagating said holographic reconstruction onto animage plane and calculating intensity at points of said image plane,producing an image of the target scene.
 43. The system of claim 37, saidfunction of said diffracted projection comprising a portion of saiddiffracted projection.
 44. The system of claim 37, said function of saidknown array pattern of said at least one original binary coded aperturemask comprising a morphed pattern of said at least one original binarycoded aperture mask.
 45. The system of claim 37, wherein: saidholographic reconstructing occurs before said correlating; saidholographic reconstructing comprises illuminating said diffractedprojection of the target scene as a hologram of said diffractedprojection to produce a pattern image; and said correlating comprisescorrelating said pattern image of said hologram with said function ofsaid known array pattern of said at least one original binary codedaperture mask, producing an image of the target scene.
 46. The system ofclaim 37, further comprising said requisite optical elements,computerized storage and computerized processing for: said function ofsaid diffracted projection comprising said diffracted projection; saidfunction of said known array pattern of said at least one originalbinary coded aperture mask comprising a prerecorded impulse responsecomprising a complex field produced by passing a wavefront through saidat least one original binary coded aperture mask onto said focal planearray; and correlating said diffracted projection with said impulseresponse to produce an image of the target scene.
 47. The system ofclaim 37, wherein: said correlating occurs before said holographicreconstructing; said correlating produces a hologram of the targetscene; and said holographic reconstructing comprises illuminating saidhologram using an array of light sources arranged in a patterncomplementary to said at least one original binary coded aperture mask,producing a holographic reconstruction of the target scene; furthercomprising said requisite optical elements, computerized storage andcomputerized processing for: focusing said holographic reconstructiononto an image plane, to produce an image of the target scene.
 48. Thesystem of claim 39, further comprising said requisite optical elements,computerized storage and computerized processing for said correlatingcomprising correlating using deconvolution.
 49. The system of claim 42,further comprising said requisite optical elements, computerized storageand computerized processing for said correlating comprising correlatingusing deconvolution.
 50. The system of claim 45, further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for said correlating comprising correlating usingdeconvolution.
 51. The system of claim 46, further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for said correlating comprising correlating usingdeconvolution.
 52. The system of claim 38, further comprising: apolarizing coded aperture mask comprising both original mask polarizerswith a first polarization orientation and complementary mask polarizerswith a second polarization orientation; and: said requisite opticalelements, computerized storage and computerized processing forrespectively separating diffracted projections produced by said originalmask polarizers and diffracted projections produced by saidcomplementary mask polarizers into said diffracted projection of thetarget scene and a complementary diffracted projection of the targetscene.
 53. The system of claim 38, further comprising: a diffractiongrating coded aperture mask comprising both original mask gratingrulings and complementary mask grating; and said requisite opticalelements, computerized storage and computerized processing forrespectively separating diffracted projections produced by said originalmask grating rulings and diffracted projections produced by saidcomplementary mask grating rulings into said diffracted projection ofthe target scene and a complementary diffracted projection of the targetscene.
 54. The system of claim 38, further comprising: a refractivecoded aperture mask comprising both original mask refractive elementsand complementary mask refractive elements; and said requisite opticalelements, computerized storage and computerized processing forrespectively separating diffracted projections produced by said originalmask refractive elements and diffracted projections produced by saidcomplementary mask refractive elements into said diffracted projectionof the target scene and a complementary diffracted projection of thetarget scene.
 55. The system of claim 38, further comprising: a mixedtransparent and reflective coded aperture mask comprising bothtransparent and reflective elements; and said requisite opticalelements, computerized storage and computerized processing forseparating diffracted projections produced by said transparent elementsand diffracted projections produced by said reflective elements intosaid diffracted projection of the target scene and a complementarydiffracted projection of the target scene.
 56. The system of claim 38,further comprising: a dynamic coded aperture mask for forming insequence, both said at least one original binary coded aperture mask andsaid complement of said at least one original binary coded aperturemask; and said requisite optical elements, computerized storage andcomputerized processing for respectively separating diffractedprojections produced by said at least one original binary coded aperturemask and diffracted projections produced by said complement of said atleast one original binary coded aperture mask into said diffractedprojection of the target scene and a complementary diffracted projectionof the target scene.
 57. The system of claim 37, said using at least onereconstructing wavefront for holographic reconstructing comprisingincreasing spatial resolution by narrowing a spectral range with afilter selected from the filter group consisting of: bandpass filtersand notch filters.
 58. The system of claim 37, said using at least onereconstructing wavefront for holographic reconstructing comprising:using a plurality of reconstructing wavefronts of different wavelengthsto focus different spectral components of the holographicreconstruction; and combining said different spectral components of saidholographic reconstruction, producing a hyperspectral image of thetarget scene.
 59. The system of claim 37, said using at least onereconstructing wavefront for holographic reconstructing comprisingapplying a change detection algorithm to said representation of thetarget scene.
 60. The system of claim 59, wherein said representation ofthe target scene is denoted by h(x,y), said change detection algorithmcomprising: producing a shifted image of said representation of thetarget scene denoted by h′(x, y), such that:h′(x,y)=h(x−a,y−b)+g(x,y), where g(x,y) denotes a difference betweensaid representation of the target scene and said shifted image of saidrepresentation of the target scene; detecting said difference g(x,y),using a first Fourier filter${{F_{1}\left( {p,q} \right)} = \frac{1}{{H\left( {p,q} \right)} + 1}},$where H(p,q) is the Fourier transform of the function h(x, y), using asecond Fourier filterF ₂(p,q)=H(p,q), and using a nonlinear filter N(·); by: starting withh′(x,y) and applying F₁(p,q) followed by N(·) followed by F₂(p,q), andthen subtracting off the shifted h′(x,y) to yield g(x,y).
 61. The systemof claim 37, wherein said at least one original binary coded aperturemask and said focal plane array are substantially parallel to oneanother.
 62. The system of claim 37, wherein said at least one originalbinary coded aperture mask and said focal plane array are substantiallynot parallel to one another.
 63. The system of claim 37, furthercomprising at least one optical element for concentrating said at leastone scene wavefront onto said focal plane array to reduce wavefrontloss.
 64. The system of claim 37, wherein said at least one originalbinary coded aperture mask is substantially planar.
 65. The system ofclaim 37, wherein said at least one original binary coded aperture maskis substantially non-planar.
 66. The system of claim 65, wherein a shapeof said at least one original binary coded aperture mask is conformal.67. The system of claim 38, further comprising: a polarimetric codedaperture mask comprising all four of: opaque elements; transparentelements polarized in a first polarization orientation; transparentelements polarized in a second polarization orientation; and unpolarizedtransparent elements transparent to said first polarization orientationand said second polarization orientation; wherein said transparentelements and said opaque elements are arranged in statisticallyindependent patterns; and said requisite optical elements, computerizedstorage and computerized processing for processing said diffractedprojection into a representation of the target scene in said first andsecond polarization orientations by correlating a function of saiddiffracted projection with a function of the pattern of said transparentelements, respectively, in said first and second polarizationorientations, and for each polarization orientation, by using at leastone reconstructing wavefront for holographic reconstructing.
 68. Thesystem of claim 37, further comprising said requisite optical elements,computerized storage and computerized processing for compensating forknown aberrations introduced by at least one of said optical elements,used between the target scene and said original binary coded aperturemask, by subtracting said known aberrations from the holographicreconstruction, to produce an aberration-compensated holographicreconstruction.
 69. The system of claim 37, further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for compensating for known aberrations introduced by at leastone of said optical elements, used between the target scene and saidoriginal binary coded aperture mask, by: remapping a digital pattern ofsaid original binary coded aperture mask, as a function of said knownaberrations; and using the remapped digital pattern for said holographicreconstructing, to produce an aberration-compensated holographicreconstruction.
 70. The system of claim 37, further comprising saidrequisite optical elements, computerized storage and computerizedprocessing for compensating for unknown aberrations introduced by atleast one of said optical elements, used between the target scene andsaid original binary coded aperture mask, by a calibration procedurecomprising: passing at least one calibration wavefront through said atleast one of said optical elements in combination with said at least oneoriginal binary coded aperture mask, onto the focal plane array,producing an aberrated diffracted projection of a source of saidcalibration wavefront; deducing a measured aberration due to said atleast one of said optical elements, from said aberrated diffractedprojection; and subtracting said measured aberration due to said atleast one of said optical elements, during processing of said diffractedprojection of the target scene, to produce an aberration-compensatedholographic reconstruction.
 71. The system of claim 37, furthercomprising said requisite optical elements, computerized storage andcomputerized processing for compensating for unknown atmosphericaberrations introduced between the target scene and said original binarycoded aperture mask, by computationally maximizing a sharpness functionduring processing of said diffracted projection of the target scene, toproduce an aberration-compensated holographic reconstruction.
 72. Thesystem of claim 37, further comprising said requisite optical elements,computerized storage and computerized processing for compensating forunknown atmospheric aberrations introduced between the target scene andsaid original binary coded aperture mask, by: subdividing at least onewavefront of the holographic reconstruction, into a plurality ofcoherent sub-wavefronts, each smaller than Fried's parameter;computationally refocusing each sub-wavefront to produce correspondingobject images; correlating said object images to calculate shifts andtilts of the corresponding sub-wavefronts, caused by the atmosphericturbulence; removing the calculated shifts and tilts to produce acorresponding plurality of corrected sub-wavefronts; and coherentlycombining said corrected sub-wavefronts, to produce anaberration-compensated holographic reconstruction.